{
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {
        "colab_type": "text",
        "id": "djUvWu41mtXa"
      },
      "source": [
        "##### Copyright 2019 The TensorFlow Authors."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "cellView": "form",
        "colab": {},
        "colab_type": "code",
        "id": "su2RaORHpReL"
      },
      "outputs": [],
      "source": [
        "#@title Licensed under the Apache License, Version 2.0 (the \"License\");\n",
        "# you may not use this file except in compliance with the License.\n",
        "# You may obtain a copy of the License at\n",
        "#\n",
        "# https://www.apache.org/licenses/LICENSE-2.0\n",
        "#\n",
        "# Unless required by applicable law or agreed to in writing, software\n",
        "# distributed under the License is distributed on an \"AS IS\" BASIS,\n",
        "# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n",
        "# See the License for the specific language governing permissions and\n",
        "# limitations under the License."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "colab_type": "text",
        "id": "NztQK2uFpXT-"
      },
      "source": [
        "# TensorBoard Scalars: Logging training metrics in Keras\n",
        "\n",
        "<table class=\"tfo-notebook-buttons\" align=\"left\">\n",
        "  <td>\n",
        "    <a target=\"_blank\" href=\"https://tensorflow.google.cn/tensorboard/scalars_and_keras\"><img src=\"https://tensorflow.google.cn/images/tf_logo_32px.png\" />在 tensorFlow.google.cn 上查看</a>\n",
        "  </td>\n",
        "  <td>\n",
        "    <a target=\"_blank\" href=\"https://colab.research.google.com/github/tensorflow/docs-l10n/blob/master/site/zh-cn/tensorboard/scalars_and_keras.ipynb\"><img src=\"https://tensorflow.google.cn/images/colab_logo_32px.png\" />Google Colab 中运行</a>\n",
        "  </td>\n",
        "  <td>\n",
        "    <a target=\"_blank\" href=\"https://github.com/tensorflow/docs-l10n/blob/master/site/zh-cn/tensorboard/scalars_and_keras.ipynb\"><img src=\"https://tensorflow.google.cn/images/GitHub-Mark-32px.png\" />在 GitHub 上查看源代码</a>\n",
        "  </td>\n",
        "  <td>\n",
        "    <a href=\"https://storage.googleapis.com/tensorflow_docs/docs-l10n/site/zh-cn/tensorboard/scalars_and_keras.ipynb\"><img src=\"https://tensorflow.google.cn/images/download_logo_32px.png\" />下载此 notebook</a>\n",
        "  </td>\n",
        "</table>"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "colab_type": "text",
        "id": "eDXRFe_qp5C3"
      },
      "source": [
        "## 概述\n",
        "\n",
        "\n",
        "机器学习总是涉及理解关键指标，例如损失 (loss) ，以及它们如何随着训练的进行而变化。 例如，这些指标可以帮助您了解模型是否[过拟合](https://en.wikipedia.org/wiki/Overfitting)，或者是否不必要地训练了太长时间。 您可能需要比较不同训练中的这些指标，以帮助调试和改善模型。\n",
        "\n",
        "TensorBoard 的** Scalars Dashboard **允许您轻松地使用简单的 API 可视化这些指标。 本教程提供了非常基本的示例，可帮助您在开发 Keras 模型时学习如何在 TensorBoard 中使用这些 API 。 您将学习如何使用 Keras TensorBoard 回调和 TensorFlow Summary API 来可视化默认和自定义标量。"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "colab_type": "text",
        "id": "dG-nnZK9qW9z"
      },
      "source": [
        "## 设置"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 1,
      "metadata": {
        "colab": {},
        "colab_type": "code",
        "id": "3U5gdCw_nSG3"
      },
      "outputs": [],
      "source": [
        "# 加载 TensorBoard notebook 插件\n",
        "%load_ext tensorboard"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 3,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 34
        },
        "colab_type": "code",
        "id": "1qIKtOBrqc9Y",
        "outputId": "cb1b3125-6f75-4fe7-ac5e-07954b5d6847"
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "TensorFlow version:  2.0.0-dev20190226\n"
          ]
        }
      ],
      "source": [
        "from __future__ import absolute_import\n",
        "from __future__ import division\n",
        "from __future__ import print_function\n",
        "\n",
        "from datetime import datetime\n",
        "from packaging import version\n",
        "\n",
        "import tensorflow as tf\n",
        "from tensorflow import keras\n",
        "\n",
        "import numpy as np\n",
        "\n",
        "print(\"TensorFlow version: \", tf.__version__)\n",
        "assert version.parse(tf.__version__).release[0] >= 2, \\\n",
        "    \"This notebook requires TensorFlow 2.0 or above.\""
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "colab_type": "text",
        "id": "6YDAoNCN3ZNS"
      },
      "source": [
        "## 配置数据用来训练回归\n",
        "\n",
        "您现在将使用 [Keras](https://tensorflow.google.cn/guide/keras) 计算回归，即找到对应数据集的最佳拟合。 （虽然使用神经网络和梯度下降[解决此类问题多此一举](https://stats.stackexchange.com/questions/160179/do-we-need-gradient-descent-to-find-the-coefficients-of-a-linear-regression-mode)，但这却是一个非常容易理解的示例.)\n",
        "\n",
        "您将使用 TensorBoard 观察训练和测试**损失 (loss) **在各个时期之间如何变化。 希望您会看到训练集和测试集损失随着时间的流逝而减少，然后保持稳定。\n",
        "\n",
        "首先，大致沿 *y = 0.5x + 2* 线生成1000个数据点。 将这些数据点分为训练和测试集。 您希望神经网络学会 x 与 y 的对应关系。"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {},
        "colab_type": "code",
        "id": "j-ryO6OxnQH_"
      },
      "outputs": [],
      "source": [
        "data_size = 1000\n",
        "# 80% 的数据用来训练\n",
        "train_pct = 0.8\n",
        "\n",
        "train_size = int(data_size * train_pct)\n",
        "\n",
        "# 创建在(-1,1)范围内的随机数作为输入\n",
        "x = np.linspace(-1, 1, data_size)\n",
        "np.random.shuffle(x)\n",
        "\n",
        "# 生成输出数据\n",
        "# y = 0.5x + 2 + noise\n",
        "y = 0.5 * x + 2 + np.random.normal(0, 0.05, (data_size, ))\n",
        "\n",
        "# 将数据分成训练和测试集\n",
        "x_train, y_train = x[:train_size], y[:train_size]\n",
        "x_test, y_test = x[train_size:], y[train_size:]"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "colab_type": "text",
        "id": "Je59_8Ts3rq0"
      },
      "source": [
        "## 训练模型和记录损失 (loss)\n",
        "\n",
        "您现在可以定义，训练和评估模型了。 \n",
        "\n",
        "要在训练时记录损失 (loss) ，请执行以下操作：\n",
        "\n",
        "1.   创建 Keras [TensorBoard 回调](https://tensorflow.google.cn/api_docs/python/tf/keras/callbacks/TensorBoard)\n",
        "2.   指定日志目录\n",
        "3.   将 TensorBoard 回调传递给 Keras' [Model.fit()](https://tensorflow.google.cn/api_docs/python/tf/keras/models/Model#fit).\n",
        "\n",
        "TensorBoard 从日志目录层次结构中读取日志数据。 在此 notebook 中，根日志目录是 ```logs/scalars``` ，后缀有时间戳的子目录。带时间戳的子目录使您可以在使用 TensorBoard 并在模型上进行迭代时轻松识别并选择训练运行。\n",
        " "
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 5,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 51
        },
        "colab_type": "code",
        "id": "VmEQwCon3i7m",
        "outputId": "edf1eca5-a759-41cf-d3f3-8ac734a06099"
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "Training ... With default parameters, this takes less than 10 seconds.\n",
            "Average test loss:  0.05271831926424056\n"
          ]
        }
      ],
      "source": [
        "logdir = \"logs/scalars/\" + datetime.now().strftime(\"%Y%m%d-%H%M%S\")\n",
        "tensorboard_callback = keras.callbacks.TensorBoard(log_dir=logdir)\n",
        "\n",
        "model = keras.models.Sequential([\n",
        "    keras.layers.Dense(16, input_dim=1),\n",
        "    keras.layers.Dense(1),\n",
        "])\n",
        "\n",
        "model.compile(\n",
        "    loss='mse', # keras.losses.mean_squared_error\n",
        "    optimizer=keras.optimizers.SGD(lr=0.2),\n",
        ")\n",
        "\n",
        "print(\"Training ... With default parameters, this takes less than 10 seconds.\")\n",
        "training_history = model.fit(\n",
        "    x_train, # input\n",
        "    y_train, # output\n",
        "    batch_size=train_size,\n",
        "    verbose=0, # Suppress chatty output; use Tensorboard instead\n",
        "    epochs=100,\n",
        "    validation_data=(x_test, y_test),\n",
        "    callbacks=[tensorboard_callback],\n",
        ")\n",
        "\n",
        "print(\"Average test loss: \", np.average(training_history.history['loss']))"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "colab_type": "text",
        "id": "042k7GMERVkx"
      },
      "source": [
        "## 使用 TensorBoard 检查损失 (loss)\n",
        "现在，启动 TensorBoard ，并指定您在上面使用的根日志目录。\n",
        "\n",
        "等待几秒钟以使 TensorBoard 进入载入界面。"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {},
        "colab_type": "code",
        "id": "6pck56gKReON"
      },
      "outputs": [],
      "source": [
        "%tensorboard --logdir logs/scalars"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "colab_type": "text",
        "id": "QmQHlG10Kpu2"
      },
      "source": [
        "<img class=\"tfo-display-only-on-site\" src=\"https://github.com/tensorflow/tensorboard/blob/master/docs/images/scalars_loss.png?raw=1\"/>"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "colab_type": "text",
        "id": "ciSIRibhRi6N"
      },
      "source": [
        "您可能会看到 TensorBoard 显示消息“当前数据集没有活动的仪表板”。这是因为尚未保存初始日志记录数据。随着训练的进行，Keras 模型将开始记录数据。TensorBoard 将定期刷新并显示您的 scalar 指标。如果您不耐烦，可以点击右上角的刷新箭头。\n",
        "\n",
        "在观看训练进度时，请注意训练和验证损失如何迅速减少，然后保持稳定。实际上，您可能在25个 epochs 后就停止了训练，因为在此之后训练并没有太大改善。\n",
        "\n",
        "将鼠标悬停在图形上可以查看特定的数据点。您也可以尝试使用鼠标放大，或选择其中的一部分以查看更多详细信息。\n",
        "\n",
        "注意左侧的 “Runs” 选择器。 “Runs” 表示来自一轮训练的一组日志，在本例中为 Model.fit() 的结果。随着时间的推移，开发人员进行实验和开发模型时，通常会有很多运行。\n",
        "\n",
        "使用 “Runs” 选择器选择特定的 Runs，或仅从训练或验证中选择。比较运行将帮助您评估哪个版本的代码可以更好地解决您的问题。\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "colab_type": "text",
        "id": "finK0GfYyefe"
      },
      "source": [
        "TensorBoard 的损失图表明，对于训练和验证，损失持续减少，然后稳定下来。 这意味着该模型的指标可能非常好！ 现在来看模型在现实生活中的实际行为。\n",
        "\n",
        "给定 (60, 25, 2), 方程式 *y = 0.5x + 2* 应该会输出 (32, 14.5, 3). 模型会输出一样的结果吗?"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 7,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 68
        },
        "colab_type": "code",
        "id": "EuiLgxQstt32",
        "outputId": "0a957477-58fe-47b4-c366-06520250b59d"
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "[[32.234306 ]\n",
            " [14.5974245]\n",
            " [ 3.0074697]]\n"
          ]
        }
      ],
      "source": [
        "print(model.predict([60, 25, 2]))\n",
        "# 理想的输出结果是: \n",
        "# [[32.0]\n",
        "#  [14.5]\n",
        "#  [ 3.0]]"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "colab_type": "text",
        "id": "bom4MdeewRKS"
      },
      "source": [
        "并不差！"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "colab_type": "text",
        "id": "vvwGmJK9XWmh"
      },
      "source": [
        "## 记录自定义 scalars\n",
        "\n",
        "如果要记录自定义值，例如[动态学习率](https://www.jeremyjordan.me/nn-learning-rate/)，该怎么办？ 为此，您需要使用 TensorFlow Summary API。\n",
        "\n",
        "重新训练回归模型并记录自定义学习率。如以下步骤所示：\n",
        "\n",
        "1.使用 ```tf.summary.create_file_writer()``` 创建文件编写器。\n",
        "2.定义自定义学习率函数。 这将传递给 Keras [LearningRateScheduler](https://tensorflow.google.cn/api_docs/python/tf/keras/callbacks/LearningRateScheduler) 回调。\n",
        "3.在学习率函数内部，使用 ```tf.summary.scalar()``` 记录自定义学习率。\n",
        "4.将 LearningRateScheduler 回调传递给 Model.fit()。\n",
        "\n",
        "通常，要记录自定义 scalars ，您需要对文件编写器使用 ```tf.summary.scalar()```。 文件编写器负责将此运行的数据写入指定的目录，并在您使用 ```tf.summary.scalar()``` 时隐式使用。"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {},
        "colab_type": "code",
        "id": "XB95ltRiXVXk"
      },
      "outputs": [],
      "source": [
        "logdir = \"logs/scalars/\" + datetime.now().strftime(\"%Y%m%d-%H%M%S\")\n",
        "file_writer = tf.summary.create_file_writer(logdir + \"/metrics\")\n",
        "file_writer.set_as_default()\n",
        "\n",
        "def lr_schedule(epoch):\n",
        "  \"\"\"\n",
        "  Returns a custom learning rate that decreases as epochs progress.\n",
        "  \"\"\"\n",
        "  learning_rate = 0.2\n",
        "  if epoch > 10:\n",
        "    learning_rate = 0.02\n",
        "  if epoch > 20:\n",
        "    learning_rate = 0.01\n",
        "  if epoch > 50:\n",
        "    learning_rate = 0.005\n",
        "\n",
        "  tf.summary.scalar('learning rate', data=learning_rate, step=epoch)\n",
        "  return learning_rate\n",
        "\n",
        "lr_callback = keras.callbacks.LearningRateScheduler(lr_schedule)\n",
        "tensorboard_callback = keras.callbacks.TensorBoard(log_dir=logdir)\n",
        "\n",
        "model = keras.models.Sequential([\n",
        "    keras.layers.Dense(16, input_dim=1),\n",
        "    keras.layers.Dense(1),\n",
        "])\n",
        "\n",
        "model.compile(\n",
        "    loss='mse', # keras.losses.mean_squared_error\n",
        "    optimizer=keras.optimizers.SGD(),\n",
        ")\n",
        "\n",
        "training_history = model.fit(\n",
        "    x_train, # input\n",
        "    y_train, # output\n",
        "    batch_size=train_size,\n",
        "    verbose=0, # Suppress chatty output; use Tensorboard instead\n",
        "    epochs=100,\n",
        "    validation_data=(x_test, y_test),\n",
        "    callbacks=[tensorboard_callback, lr_callback],\n",
        ")"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "colab_type": "text",
        "id": "pck8OQEjayDM"
      },
      "source": [
        "查看 TensorBoard "
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {},
        "colab_type": "code",
        "id": "0sjM2wXGa0mF"
      },
      "outputs": [],
      "source": [
        "%tensorboard --logdir logs/scalars"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "colab_type": "text",
        "id": "GkIahGZKK9I7"
      },
      "source": [
        "<img class=\"tfo-display-only-on-site\" src=\"https://github.com/tensorflow/tensorboard/blob/master/docs/images/scalars_custom_lr.png?raw=1\"/>"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "colab_type": "text",
        "id": "RRlUDnhlkN_q"
      },
      "source": [
        "使用左侧的 “Runs” 选择器，请注意您运行了 ```<timestamp>/metrics```。 选择此运行将显示一个 \"learning rate\" 图，您可以在此运行过程中验证学习率的进度。\n",
        "\n",
        "您还可以将此运行的训练和验证损失曲线与您以前的运行进行比较。"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "colab_type": "text",
        "id": "l0TTI16Nl0nk"
      },
      "source": [
        "模型会输出什么呢？"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 10,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 68
        },
        "colab_type": "code",
        "id": "97T4vT3QkQJH",
        "outputId": "fe4614dc-f58d-4804-9e48-d62dcbb9a8ea"
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "[[32.234013 ]\n",
            " [14.5973015]\n",
            " [ 3.0074618]]\n"
          ]
        }
      ],
      "source": [
        "print(model.predict([60, 25, 2]))\n",
        "# 理想的输出结果是: \n",
        "# [[32.0]\n",
        "#  [14.5]\n",
        "#  [ 3.0]]"
      ]
    }
  ],
  "metadata": {
    "colab": {
      "collapsed_sections": [],
      "name": "scalars_and_keras.ipynb",
      "provenance": [],
      "toc_visible": true,
      "version": "0.3.2"
    },
    "kernelspec": {
      "display_name": "Python 3",
      "name": "python3"
    }
  },
  "nbformat": 4,
  "nbformat_minor": 0
}
